Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support MariaDB database #375

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Support MariaDB database #375

wants to merge 1 commit into from
+488 −0

Conversation

HugoWenTD
Copy link

@HugoWenTD HugoWenTD commented Oct 9, 2024 •
edited
Loading

MariaDB introduced vector support in version 11.7, enabling MariaDB
Server to function as a relational vector database.
https://mariadb.com/kb/en/vectors/

Now add support for MariaDB server, verified against MariaDB server
of version 11.7.1:

  • Support MariaDB vector search with HNSW algorithm, support filter
    search.
  • Support index and search parameters:
    • storage_engine: InnoDB or MyISAM
    • M: M parameter in MHNSW vector indexing
    • ef_search: minimal number of result candidates to look for in the
      vector index for ORDER BY ... LIMIT N queries.
    • max_cache_size: Upper limit for one MHNSW vector index cache
  • Support CLI of vectordbbench mariadbhnsw.

@sre-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: HugoWenTD
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@HugoWenTD
Copy link
Author

/assign @XuanYang-cn

@liliu-z
Copy link
Collaborator

liliu-z commented Oct 10, 2024

/assign @alwayslove2013

@vuvova
Copy link

vuvova commented Oct 10, 2024

Please wait with merging this until MariaDB vector support is actually released (it's expected to happen within a month). The syntax and configuration settings can still change from the dev branch

@HugoWenTD
Support MariaDB database
83f49c4 
MariaDB introduced vector support in version 11.7, enabling MariaDB
Server to function as a relational vector database.
https://mariadb.com/kb/en/vectors/

Now add support for MariaDB server, verified against MariaDB server
of version 11.7.1:

- Support MariaDB vector search with HNSW algorithm, support filter
  search.
- Support index and search parameters:
   - storage_engine: InnoDB or MyISAM
   - M: M parameter in MHNSW vector indexing
   - ef_search: minimal number of result candidates to look for in the
                vector index for ORDER BY ... LIMIT N queries.
   - max_cache_size: Upper limit for one MHNSW vector index cache
- Support CLI of `vectordbbench mariadbhnsw`.
@HugoWenTD
Copy link
Author

Updated the commit to adapt syntax in MariaDB 11.7.1 and support filtered search. The small test set works well. Now running some bigger tests to see what I can get.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhuwenxing zhuwenxing zhuwenxing left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
hold
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@HugoWenTD @sre-ci-robot @liliu-z @vuvova @zhuwenxing @alwayslove2013